REGO: Rank-based Estimation of Renyi Information using Euclidean Graph Optimization
نویسندگان
چکیده
We propose a new method for a nonparametric estimation of Rényi and Shannon information for a multivariate distribution using a corresponding copula, a multivariate distribution over normalized ranks of the data. As the information of the distribution is the same as the negative entropy of its copula, our method estimates this information by solving a Euclidean graph optimization problem on the empirical estimate of the distribution’s copula. Owing to the properties of the copula, we show that the resulting estimator of Rényi information is strongly consistent and robust. Further, we demonstrate its applicability in image registration in addition to simulated experiments.
منابع مشابه
Estimation of Renyi Entropy and Mutual Information Based on Generalized Nearest-Neighbor Graphs
We present simple and computationally efficient nonparametric estimators of Rényi entropy and mutual information based on an i.i.d. sample drawn from an unknown, absolutely continuous distribution over R. The estimators are calculated as the sum of p-th powers of the Euclidean lengths of the edges of the ‘generalized nearest-neighbor’ graph of the sample and the empirical copula of the sample r...
متن کاملTesting Exponentiality Based on Renyi Entropy of Transformed Data
In this paper, we introduce new tests for exponentiality based on estimators of Renyi entropy of a continuous random variable. We first consider two transformations of the observations which turn the test of exponentiality into one of uniformity and use a corresponding test based on Renyi entropy. Critical values of the test statistics are computed by Monte Carlo simulations. Then, we compare p...
متن کاملAn Effective Approach for Robust Metric Learning in the Presence of Label Noise
Many algorithms in machine learning, pattern recognition, and data mining are based on a similarity/distance measure. For example, the kNN classifier and clustering algorithms such as k-means require a similarity/distance function. Also, in Content-Based Information Retrieval (CBIR) systems, we need to rank the retrieved objects based on the similarity to the query. As generic measures such as ...
متن کاملPotential Statistical Evidence in Experiments and Renyi Information
Recently Habibi et al. (2006) defined a pre-experimental criterion for the potential strength of evidence provided by an experiment, based on Kullback-Leibler distance. In this paper, we investigate the potential statistical evidence in an experiment in terms of Renyi distance and compare the potential statistical evidence in lower (upper) record values with that in the same number of ii...
متن کاملRelations between Renyi Distance and Fisher Information
In this paper, we first show that Renyi distance between any member of a parametric family and its perturbations, is proportional to its Fisher information. We, then, prove some relations between the Renyi distance of two distributions and the Fisher information of their exponentially twisted family of densities. Finally, we show that the partial ordering of families induced by Renyi dis...
متن کامل